Accelerated Variance Reduced Block Coordinate Descent

Algorithms with fast convergence, small number of data access, and low periteration complexity are particularly favorable in the big data era, due to the demand for obtaining highly accurate solutions to problems with a large number of samples in ultra-high dimensional space. Existing algorithms lack at least one of these qualities, and thus are inefficient in handling such big data challenge. In this paper, we propose a method enjoying all these merits with an accelerated convergence rate O( 1 k2 ). Empirical studies on large scale datasets with more than one million features are conducted to show the effectiveness of our methods in practice.